oearcning rMU 


\/ L — 


PATENT ABSTRACTS OF JAPAN 


( 1 1 )Publication number : 1 0-208073 

(43)Date of publication of application : 07.08.1 998 


(51)Int.CI. 


G06T 15/00 
H04N 13/02 


(21) Application number : 09-005243 

(22) Date of filing: 16.01.1997 


(71) Applicant 

(72) Inventor : 


HITACHI LTD 

MINAGAWA TAKESHI 
MORIYA TOSHIO 
TAKEDA HARUO 


13 


Hi 


(54) VIRTUAL REAUTY CREATING DEVICE 
(57)Abstract: 

PROBLEM TO BE SOLVED: To harmonize a real world 
and a virtual world in a natural form anywhere by 
synthesizing data inputted from a moving body with data 
provided from a data providing means correspondingly to 
movement of the moving body. 

SOLUTION: A data input device 10 is provided with a 
moving robot 1 1 and a data input means 1 2 and is 
controlled by an input signal from a communication 
means 62 and uses a communication means 63 to output 
signals from the moving robot 1 1 and the data input 
means 12. A data providing device 20 is provided with a 
data providing means 21. A data processing and 
synthesizing device 30 is provided with a data 
processing and synthesizing means 31 and uses the data 
processing and synthesizing means 31 to 
process/synthesize and output the input signals from a 
communication means 64 and the data providing means 
21. A data output device 40 uses a data output means 
41 to output the input signal from the data processing 

and synthesizing means 31. A controller 50 of the data input device generates a control signal, 
which controls the moving robot 1 1 and the data input means 12, by a control means 51 and 
uses a communication means 61 to output it. 
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* NOTICES * 

JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

1 This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 


CLAIMS 


[Claim(s)] 

[Claim 1] The data synthesizer unit characterized by to have the data supply equipment which 
can supply data other than the data inputted from the mobile in which a data input is possible, 
the data which matched with migration of said mobile the data inputted from said mobile, and 
were supplied from said data supply means, and data processing and the synthesizer unit 
compound, and data-output equipment which output the data compounded with said data 
processing and synthesizer unit. 

[Claim 2] The data inputted from said mobile in a data synthesizer unit according to claim 1 are a 
data synthesizer unit characterized by being the image data which said mobile picturizes. 
[Claim 3] The data synthesizer unit characterized by performing delivery of the data between 
said data processings and synthesizer units, and data output means on radio using means of 
communications in a data synthesizer unit according to claim 1 . 

[Claim 4] The merge approach characterized by matching with reception the 2nd data other than 
the 1 st data inputted from the mobile, and said data, matching said 1 st data with migration of 
said mobile, and compounding said 2nd data. 


[Translation done.] 
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DETAILED DESCRIPTION 

[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention relates to the data synthesizer unit and the merge 
approach of compounding separate data. It is related with the equipment which realizes virtual 
reality which used the mobile robot especially. It is related with realizing an image (virtual reality) 
in detail based on the image inputted by the mobile robot. 
[0002] 

[Description of the Prior Art] There is the following in the conventional technique for creating 
virtual reality. First, an object is photoed to the virtual reality listing device "which sets the 
leading role as the target of the real world" in the room beforehand made for photography, and 
there is equipment which embeds and outputs the object to the virtual world made beforehand in 
it. Generally this is called virtual studio. Or I have human being who becomes a candidate for 
photography learn a marker beforehand, a motion of the marker is photographed with two or 
more cameras, and there is a technique of incorporating natural actuation as digital data, by 
following the motion of those markers on analysis equipment. Generally this is called a motion 
capture technique. 

[0003] Next, how many three-dimension configurations of a photography environment are 
restored from that image, it comes out, various actuation is carried out, and there is a technique 
for which a three-dimension CG character is not embedded in the environment and which 
outputs the image among the virtual reality creation techniques "which embeds the element of a 
virtual world based on an on-the-spot photo." Generally this is called an on-the-spot photo CG 
composition technique. 

[0004] Furthermore, a robot is moved with the parameter defined beforehand and the equipment 
which takes a photograph is one of things using a mobile robot. This is used by the 
cinematography etc. and, generally is called the motion control camera. 
[0005] 

[Problem(s) to be Solved by the Invention] With these techniques, since it was difficult, starting 
automatically the object embedded to a virtual world from the image of the real world needed 
rooms of special conditions, such as the blue back's room, in said virtual studio, in order to make 
logging easy. Therefore, virtual reality was not able to be experienced when it was not in the 
room. Moreover, although the camera is installed in the specific location of the room and 
include-angle accommodation can be performed, since migration of a location cannot be 
performed, if distance with the camera for photography separates, the quality of an image will 
deteriorate. Moreover, in a motion capture technique, there is troublesomeness that the 
candidate for photography must learn a marker beforehand, and since it must move learning a 
marker, there is a problem that virtual reality cannot be experienced automatically, 
unconsciously. 

[0006] Moreover, in an on-the-spot photo CG composition technique, although there is an 
advantage with the quality of an image "a location is not chosen", "maintainable", the character 
of a virtual world only moves and there is no interaction of the real world and a virtual world. 
Moreover, about a point without the interaction of the real world and a virtual world, a motion 
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control camera is the same as that of an on-the-spot photo CG composition technique. 
[0007] This invention aims at offering the equipment which realizes reconciliation of the natural 
form of the real world and a virtual world, without choosing a location to the above technical 
problem. Namely, in order not to need the specific room etc. for use of equipment but to 
experience virtual reality, the preparation by the side of a user, such as giving a marker, is 
unnecessary, the quality of the input from an input means can be maintained while it has been 
high, and it aims at offering the equipment which does an interaction and can suit between the 
real world and a virtual world further. 
[0008] 

[Means for Solving the Problem] This invention was considered as the following configurations, in 
order to solve the aforementioned problem. It is the data synthesizer unit characterized by to 
have the data supply equipment which can supply data other than the data inputted from the 
mobile in which a data input is possible, the data which matched with migration of said mobile the 
data inputted from said mobile, and were supplied from said data supply means, and data 
processing and the synthesizer unit compound, and data-output equipment which output the 
data compounded with said data processing and synthesizer unit Moreover, it is the merge 
approach characterized by matching with reception the 2nd data other than the 1st data 
inputted from the mobile, and said data, matching said 1 st data with migration of said mobile, and 
compounding said 2nd data. 

[0009] More specifically, it is as follows. It had the data supply equipment characterized by to 
have two of a mobile robot, the data entry unit equipped with a data input means are attached to 
it, a data input means or means of communications, a storage means, or sorts or more of they, 
data processing and a synthesizer unit, a data-output means, and the control means of a data 
entry unit, and the configuration that sequential-process it, and data processing and a 
synthesizer unit compound the data which obtained from said input device and said data supply 
equipment, and outputs to an output means took. 

[0010] When the data input means is attached to the mobile robot, since a data input means 
becomes possible [ moving freely ], a data input can be performed without calling at a specific 
location, and the quality of the data inputted since a data entry unit is controllable by the control 
means of a data entry unit can be maintained. Furthermore, by offering the means for "the 
element of a virtual world is embedded to the data expressing the real world", and "processing 
sequentially and outputting the inputted data" in addition to the ability moving a data input 
means freely, between the real world and a virtual world, an interaction can be done and it can 
suit. Moreover, the preparation by the side of a user becomes unnecessary by offering the 
means which embeds the element of a virtual world based on the real world. 
[0011] 

[Embodiment of the Invention] Hereafter, the gestalt of operation of this invention is explained to 
a detail. Drawing 1 is the functional block diagram showing one example of this invention. In 
drawing 1 , a data entry unit 10 is equipped with a data input means 12 to be attached to a 
mobile robot 1 1 and a robot 11, can perform delivery of means of communications 62 and 63 and 
a signal (data), is controlled by the input signal from means of communications 62. and outputs 
the signal from a mobile robot 1 1, and the signal from the data input means 12 using means of 
communications 63. Data supply equipment 20 seems to have the data supply means 21. Data 
processing and a synthesizer unit 30 are equipped with data processing / composition means 31. 
using data processing / composition means 31, processes and compounds the input signal from 
means of communications 64, and the input signal from the data supply means 20, and outputs 
them. Data output equipment 40 is equipped with the data output means 41, and outputs the 
input signal from data processing / composition means using the data output means 41. The 
control device 50 of a data entry unit generates the control signal which controls a mobile robot 
11 and the data input means 12 by the control means 51 so that it may be equipped with the 
control means 51 of a data entry unit and can perform delivery of means of communications 61 
and a signal, and it outputs the signal using means of communications 61. 
[0012] In addition, in this example, it connects by the approach which used means of 
communications between a control device 50 and a data entry unit 10 and between a data entry 
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unit 10. and data processing and a synthesizer unit 30. Although the configuration which carries 
out direct continuation with a signal line was taken between data supply equipment 20. and data 
processing and a synthesizer unit 30 and between data processing and a synthesizer unit 30. and 
data output equipment 40 That delivery of a signal is just performed in connection between 
these equipments, can also carry out direct continuation between equipment 50 and equipment 
10 or of between equipment 10 and equipment 30 with a signal line, and Between equipment 20 
and equipment 30 or between equipment 30 and equipment 40 are also connectable by the 
approach using means of communications. Moreover, the approach of carrying out direct 
continuation with a signal line and the approach of connecting using means of communications 
may be used together. 

[0013] Furthermore, although data supply equipment 20. data processing and a synthesizer unit 
30. and the control device 50 prepared in this example, respectively, when data processing and a 
synthesizer unit 30 are equipped with a data supply means, in the equipment by this invention, a 
configuration which serves both as a part or all of a role of data supply equipment 20. and the 
configuration which serves both as a part or all of a role of a control device 50 when data 
processing and a synthesizer unit 30 are equipped with a control means are also possible. In 
taking a configuration data processing and whose synthesizer unit 30 are especially equipped 
with a data supply means, it is cautious of the ability of the configuration which includes a data 
input means as a component of data processing and a synthesizer unit to be considered. 
[0014] Drawing 2 is the equipment configuration Fig. showing one example at the time of 
constituting the configuration of drawing 1 from a concrete device. In drawing 2 , a mobile robot 
1 1 0 has the command decoder 111, the migration robot control device 1 1 2. a drive system 1 1 3, a 
sensor 114, the video camera controlling mechanism 115, the microphone controlling mechanism 
116, A/D converters 117 and 118, communication boards 61 1 and 621, and 622 and 623. and the 
video camera 121 and the microphone 122 are attached. Analyzing the control signal inputted 
through means of communications 61 1 by the command decoder 111, the command decoder 1 1 1 
sends a suitable control signal to each controlling mechanisms 112, 115, and 1 1 6. The migration 
robot control device 1 1 2 controls a drive system 1 1 3 and a sensor 1 1 4 based on the control 
signal sent from the command decoder 111. 

[0015] As a drive system 113, the device in which repositioning within the mobile robot 110 of 
the video camera 121 besides a mobile robot's 110 migration device or modification of camera 
angle, repositioning within the mobile robot 1 10 of a microphone 122, etc. are performed is also 
included. Moreover, the data obtained from the sensor 114 are outputted from a communication 
board 623. The video camera controlling mechanism 115 performs control of close / OFF of the 
power source of a video camera 121, a focal distance, and a diaphragm based on the control 
signal sent from the command decoder 111. The signal inputted from the video camera 121 is 
changed into a digital signal by A/D converter 117, and is outputted from a communication board 
621. Moreover, the microphone controlling mechanism 116 controls close/OFF of the power 
source of a microphone 122 based on the control signal sent from the command decoder 111. 
[0016] The signal inputted from the microphone 122 is changed into a digital signal by A/D 
converter 1 1 8, and is outputted from a communication board 622. The signal which was 
outputted from communication boards 621, 622, and 623. and was inputted from communication 
boards 641, 642. and 643, respectively is outputted to a loudspeaker 412 using a screen 411 or a 
sound board 315 using a graphics board 314, after using the data processing program 3121 and 
the merge program 3122 with the CG model 21 1 of a virtual character, and the voice data 212 of 
a virtual character, being processed and compounded with arithmetic and program control 311 
and being further changed into a suitable form by the data output program 3123. 
[0017] Here, the input from communication boards 641. 642, and 643 is controlled by arithmetic 
and program control 311 using the communication control program 3124. Moreover, the CG 
model 21 1. voice data 212, and programs 3121, 3122. 3123, and 3124 are read from external 
storage 313 in main storage 312 if needed. A mobile robot's controller (human being) 81 controls 
a mobile robot 1 1 0 by the keyboard 51 6 or the mouse 517, looking at the image of the 
circumference environment of the mobile robot 110 inputted from a video camera 511 with a 
monitor TV 5121. Using the data 5141 for migration robot controls, the data 5142 for video 
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camera control, and the data 5143 for microphone control, based on the input -> control-code 
conversion program 5144, the input from a keyboard 516 and a mouse 517 is changed into a 
control code by arithmetic and program control 513, and is outputted from a communication 
board 681 with it. Here, the communication board 681 is controlled by arithmetic and program 
control based on the communication control program 5145. 

[0018] Moreover, the data 5141, 5142, and 5143 for control and programs 5144 and 5145 are 
read into main storage 514 from external storage 515 if needed. In addition, a signal is delivered 
on radio between communication boards 623 and 643 between communication boards 622 and 
642 between communication boards 621 and 641 between communication boards 611 and 681. 
Moreover, although external storage 313 was prepared in said example, if all data areas, the 
program fields, and working areas can secure on main storage, it is also possible to omit external 
storage 313. 

[0019] Moreover, in said example, although the keyboard 516 and the mouse 517 were used as a 
user interface of a control means, the user interface of the control means in this invention is not 
restricted to these that what is necessary is just what can create the input -> controhcode 
conversion program 5144. For example, it is also possible to use a data glove, a joy stick, etc. 
Moreover, although the controller 81 was controlling the data entry unit 10 based on the 
information from a monitor TV 5121 in said example, the information from these devices can also 
be used in the case of control of a data entry unit 10 by newly preparing a monitor TV 51 22 and 
a loudspeaker 5181, branching and outputting the output from data processing and a synthesizer 
unit 30 also to a monitor TV 5122 and a loudspeaker 5181. 

[0020] Moreover, although the controller 81 was performing only control of a data entry unit 10 
in said example, it is using what was equipped with the data input means as data processing and 
a synthesizer unit 30 with the equipment in this invention, and it can add a certain operation also 
about an output, a controller 81 controlling a data entry unit 10. 

[0021] Drawing 3 and drawing 4 are drawings showing one example in the case of using the 
equipment which used as the base the configuration shown by drawing 2 at the party hall of a 
stand-up meal format. Drawing 3 is plant layout drawing showing one example of the data entry 
unit in this case, and drawing 4 is the schematic diagram showing one example in the party hall in 
this case. 

[0022] In drawing 3 , a data entry unit 10 attaches a video camera 121, a microphone 122, an 
arm 131, and the Bon Festival 132 to the mobile robot 1 10 of drawing 2 . At this time, a sensor 
114 presupposes that it is the sensor group which can detect four data of the sense within the 
location within a mobile robot's passing speed, the sense, and the mobile robot 110 of a video 
camera 121, and the mobile robot 110 of a video camera 121. In addition, the video camera 121 
calibrates beforehand. In drawing 4 ,101 and 102 are the same as the data entry unit 10 of 
drawing 3 , and these are both controlled by the controller 81 using the control unit 501 and the 
control unit 502 from control room 711. 

[0023] The image and voice which were inputted from the camera 1211 and microphone 1221 of 
a data entry unit 101 are compounded by the input from data supply equipment 201, and data 
processing and a synthesizer unit 301, and are outputted from a screen 411 1 and a loudspeaker 
4121. Similarly, the image and voice which were inputted from the camera 1212 and microphone 
1222 of a data entry unit 102 are compounded by the input from data supply equipment 202, and 
data processing and a synthesizer unit 302, and are outputted from a screen 41 12 and a 
loudspeaker 4122. Both the inputs from the data supply equipments 201 and 202 consist of the 
CG model 21 1 of a virtual character, and the voice data 212 of a virtual character. Moreover, the 
installation of data processing and a synthesizer unit 30 does not ask. The situation in the hall is 
projected on the monitor TV 5121 installed in control room 71 1 with the video camera 51 1 
installed in the hall, and a controller 81 can control data entry units 101 and 102, looking at the 
image. 

[0024] In drawing 4 , the image acquired with the video camera 1211 is compounded with a 
virtual world in the form where transpose a mobile robot 101 to the virtual character 21 1 1, and it 
transposes a mobile robot 102 to the virtual character 2112, and is outputted to a screen 41 1 1. 
Similarly, the image acquired with the video camera 1212 is compounded with a virtual world in 
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the form where transpose a mobile robot 101 to the virtual character 21 1 1, and it transposes a 
mobile robot 102 to the virtual character 21 12, and is outputted to a screen 41 12. Similarly, the 
input from a microphone 1221 is compounded with voice data 2121, and the input from a 
microphone 1222 is compounded by the loudspeaker 4121 with voice data 2122, and is outputted 
to it at a loudspeaker 4122. Here, depth information is searched for from the image of each video 
camera by the time series image processing, and after embedding what removed the mobile robot 
from the obtained three-dimension field, and processed the three-dimension CG model 211 of a 
virtual character, the approach of changing into two-dimensional data again is used for 
composition of an image. Moreover, mixing can perform composition of speech information. 
[0025] In addition, although control room 71 1 was installed in the second floor in the party hall in 
said example, as for control room 71 1, it is arbitrary where control room is installed by not being 
attached to the party hall. Moreover, depending on the gestalt of a control means, control room 
71 1 is also omissible. Moreover, in said example, it is not necessary to restrict a data entry unit 
10 to what was illustrated that what is necessary is just to have an input means to be attached 
to a mobile robot and it. Although it had the sensor 114 and the communication board 623 in said 
example especially, this is careful of an abbreviation being possible depending on not the 
indispensable component of a data entry unit but the processing / composition approach of data. 

[0026] Moreover, although the data input means 12 were only a video camera 121 and a 
microphone 122 in said example, the data input means of this invention is not restricted to 
these. Similarly, although the data output means 41 were only a screen 41 1 and a loudspeaker 
412 in said example, the data output means of this invention is not restricted to these. For 
example, a liquid crystal display 413 is added as the pen tablet 123 and a data output means as a 
data input means. It is made for the image of a liquid crystal display 413 to be reflected in the 
background of the pen tablet 1 23 here. If the output of data processing and a synthesizer unit 30 
is branched and it is made for the same image as a screen 41 1 to be reflected in a liquid crystal 
display 413, an illustration can be drawn on a screen 41 1 in collaboration, and it can also scribble 
on the face of the virtual character on a screen 411. 

[0027] Especially the signal that a data input means and a data output means treat is noticed 
about it not being what is restricted to a picture signal and a sound signal. For example, the arm 
133 which has structure which can grasp an object instead of an arm 131 is attached to a mobile 
robot 1 10, a pressure sensor is prepared for him as an input means, and the data for control of 
an arm 133 are got from data supply equipment 20. Based on the data for control of the arm 133 
which obtained the input from a pressure sensor from data supply equipment 20, it sets to data 
processing and a synthesizer unit 30. With processing and compounding and outputting a suitable 
control code to an arm 133 Hands [ character / virtual ] can be shaken including special effect, 
such as "wearing the face which seems to be painful and grasping again strongly (this expression 
being outputted to a screen 41 1), if it grasps strongly", and can also be shaken in the place 
distant through two mobile robots. In addition, delivery of the signal between data processing and 
a synthesizer unit 30, and the arm 133 as one of the output means is performed on radio using 
means of communications in this case. 

[0028] Furthermore, although main storage was used as data supply equipment in said example, 
the data supply means in this invention is not restricted to main storage that what is necessary 
is just to have a data input means, means of communications, storage means, or those two sorts 
or more. Therefore, as an output of data supply equipment 20, the output of the data supply 
equipment 20 which can be put on this invention although the output (when it has means of 
communications) of the various processors besides the data (when it has a storage means) 
prepared beforehand and this equipment, a user s input, etc. can be considered (when it has a 
data input means) is not restricted to these. Furthermore, although the data outputted from data 
supply equipment were CG model of a virtual character, and the voice data of a virtual character, 
they are not restricted to these here as data outputted from data supply equipment, 
[0029] Furthermore, in said example, it is not necessary to restrict data processing and a 
synthesizer unit to what was illustrated that what is necessary is just what acquires the input 
from a data entry unit 10, and the input from data supply equipment 20, processes and 


http:/ / www4.ipdl.ncipi.go.jp/cgi-bin/tran.web.cgi„eije 


2005/03/09 


compounds them, and can be outputted to data output equipment 40 using means of 
communications. Furthermore, in said example, it does not restrict to what was illustrated that 
what is necessary is just what can control a required thing among the components of a data 
entry unit as a control device of a data entry unit. For example, if the power source of a video 
camera 121 and a microphone 122 is made the same with a mobile robot's 110 power source in 
said example and control of a video camera 121 and a microphone 122 will not be performed 
other than close/OFF of a power source, controlling mechanisms 1 15 and 116, the data 5142 
and 5143 for control, and the command decoder 1 1 1 are omissible. Moreover, when the hall can 
be overlooked from control room 71 1 like said example for example, it is also possible to control 
looking at the situation in the direct hall without using a video camera 51 1 and a monitor TV 
5121. 

[0030] Furthermore, means of communications which receives the output from a data entry unit 
10 can be added as a component of a control device 50, and control of a data entry unit 10 can 
also be made easier using the data fed back from the data entry unit 10. Moreover, as an object 
which a control device 50 controls, although there are a motion of a mobile robot 11, various 
setup (for example, the focal distance of a camera, a diaphragm, etc.) of the location in the 
mobile robot 1 1 of the data input means 12 and a posture, and a data input means, etc., the 
controlled system of a control device 50 is not restricted to these, for example. Moreover, 
although the number of data entry units 10 was two in said example, the number of data entry 
units 10 cannot be restricted to two sets, can also be reduced to one set, and can also be used. 
[ many ] For example, in said example, it is possible to carry a drink etc. on the Bon Festival 132 
of that data entry unit 1 0 how many sets, and to also make it move about the inside of the party 
hall instead of a waiter. 

[0031] Furthermore, although it was one set at a time, it is not necessary to restrict the video 
camera 121 and microphone 122 which are attached to an input device 10 in said example to one 
set, and these can use them, respectively as required. For example, the stereo input of an image 
and voice is attained by using two sets at a time, respectively. 

[0032] Furthermore, although the number of data supply equipment 20, the number of data 
processing and synthesizer units 30, the number of data output equipment 40, and the number of 
the control devices 50 of a data entry unit were in agreement with the number of data entry 
units 10 in said example, it cannot be dependent on the number of data entry units 10, and only 
a required number can be respectively used for the number of these equipments in this 
invention. For example, in said example, a data entry unit is four sets, 101, 102, 103, and 104, and 
data supply equipment, and data processing and a synthesizer unit are possible also for the 
configuration of [ equipment / one set (20 and 30) and / data output ] one set (50) in two sets, 
41 1 1 and 41 12, and the control unit of a data entry unit respectively. If a screen is used as a 
video camera and a data output means as a data input means in this case, it can be said that the 
output concerning a data entry unit 104 to the left half of a screen 41 12 in the output 
concerning a data entry unit 103 in the output concerning a data entry unit 102 in the output 
about a data entry unit 101 is displayed on the left half of a screen 41 1 1 in the right half of a 
screen 41 12 in the right half of a screen 41 1 1, respectively. Moreover, it is also possible to 
change automatically the output about which input unit is displayed for every fixed time amount, 
and if data processing and a synthesizer unit 30 are equipped with the input means, it is also 
possible to change which output a controller 81 aims at various effectiveness and is displayed. 
[0033] Furthermore, although the number of the party halls was one in said example, this 
invention is not restricted to this. A data entry unit is movable, and since it is possible to input 
the real world in some locations, it is also possible to carry out 2 (701 702) preparation of the 
same party hall as said example, for example, to project the situation in the party hall 701 on a 
screen 41 1 1, and to project the situation in the party hall 702 on a screen 41 12, Moreover, 
although the calibration of a camera was performed beforehand and the sensor 110 was attached 
to the mobile robot in order to acquire depth information from an image in said example, the 
configuration change of the above-mentioned preliminary data input unit is not restricted to 
what was illustrated. 

[0034] Furthermore, the three-dimension configuration presumption technique from an image is 
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not restricted to the aforementioned time series image processing. For example, if two sets of 
video cameras are used as a data input means, depth information can be acquired by the stereo 
image processing. For example, the data in the party halls, such as a location of the top view and 
obstructions of the hall (a table, chair, etc.), are obtained beforehand, and it stores in main 
storage. Furthermore, give data processing and a synthesizer unit from data supply equipment, or 
A transmitter is attached to obstructions, such as a table, a chair, etc. at the hall, and the 
receiver is attached in a mobile robot 1 10. Give more exact location and include-angle 
information to data processing and a synthesizer unit, or If the data which attached the 
transmitter to the obstruction and mobile robot at the hall, attached the receiver to head lining 
in the hall, and were obtained from the receiver are given to data processing and a synthesizer 
unit through data supply equipment, the presumed precision of restoration of depth information 
can be raised. 

[0035] Furthermore, although processing "restored depth information, embedded the three- 
dimension model of a virtual character, and it was changed and outputted to two-dimensional 
data" about processing / composition processing to the input of a picture signal in said example 
was performed, processing / composition processing to the input of the picture signal in this 
invention is not restricted to this. For example, you may decide to perform processing which 
draws a two-dimensional virtual character a top and makes it the suitable location of the 
inputted image if needed. Furthermore, it is also possible to, perform processing about the 
shadow of a mobile robot and a virtual character for example, or to remove a mobile robot's 
image, to interpolate a lack part from other images, and to project the image in "the party hall in 
which a mobile robot is not present" 

[0036] Furthermore, although composition of the sound signal in processing and the synthesizer 
unit 30 of data only performed mere mixing in said example, it is using a vocal quality conversion 
technique, and it is also possible to process a user's vocal quality. For example, in said example, 
users 821 and 822 are projected on the left half of a screen 411 1, and a right half by rise, 
respectively, and "a mimicry duet" can be realized, if users' 821 and 822 vocal quality is 
processed using the vocal quality data of the singer who stored in main storage 312 beforehand 
and it outputs to loudspeakers 4121 and 4122. 

[0037] Moreover, although the virtual character was used only as replacement of a mobile robot 
in said example, use of a virtual character is not restricted to this. For example, it compounds on 
the image into which how many persons of that virtual character were inputted, and it can also 
output to a screen 41 1 as a "sponsor", can compound on the image into which how many 
persons of that virtual character were inputted, and can also output to a screen 41 1 as "a 
participant in a party." The participant 82 and virtual character of a party are able "to play a 
janken" in the image projected on the screen 41 1. [ that the participant 82 and virtual character 
of a party sing a duet by this in the image projected on the screen 41 1, ] In addition, as a CG 
model of a virtual character, if a motion of the three-dimension model of the two-dimensional 
character drawn with the draw tool and a fictitious character, a real person's three-dimension 
model, and the real person that incorporated with the motion capture technique etc. is data 
which can be treated with data processing and a synthesizer unit 30, it will not ask the 
classification. 

[0038] Moreover, although the data entry unit 10 was controlled by the controller 81 using the 
control device 50 in said example, control of the data entry unit in this invention is not restricted 
to remote control by human being. For example, it is also possible to build into a mobile robot 
110 the main storage which memorized the various data for control and the program for control, 
the arithmetic and program control which generates the code for control using those data and 
programs, and the sensor which is required for control, and to carry out autonomous action to a 
data entry unit 10. 

[0039] According to the above example, the equipment by this invention can also use 
**********, such as the party hall, without a problem by making a data entry unit into "a mobile 
robot and a data input means." Moreover, since a data input means can move about freely with a 
mobile robot, unlike the camera of the conventional location cover half, approaching the target 
object and obtaining a high quality input or performing the input by various camera angle (vertical 
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and horizontal migration, modification of an elevation angle), and an object to photograph can be 
chased after. 

[0040] By moreover, the plan "to embed the element of a virtual world to the data expressing 
the real world ' The world of virtual reality can be experienced without being conscious since 
preparation of giving a marker in the user side who becomes a candidate for photography is 
unnecessary. Furthermore, since a "virtual world ' can be embedded as required based on the 
Veal world", the virtual reality more near reality is realizable, and if it is the case where real time 
nature is thought as important, further Real time nature can be raised by managing processing / 
composition processing in which the equipment engine performance was balanced. Furthermore, 
since real time nature improves by reducing the amount of the processing which processing and 
composition take compared with the case "where the element of the real world cut out from the 
on-the-spot photo based on the virtual world is embedded", the sense of reality at the time of 
doing an interaction mutually by the real world and the virtual world improves. Furthermore, the 
fitting image which does not have conflict in points, such as a context, the Perth effectiveness, 
and a shadow, depending on the processing at the time of embedding a virtual character on the 
image of the real world is acquired, and more real virtual reality can be created. 
[0041] Moreover, it becomes possible to synchronize action in the real world, and action by the 
virtual world by the plan "to sequential process and compound an input and to output it." 
Furthermore, since an input means becomes possible [ always existing near the user ] by 
attaching an input means to a mobile robot, it is realizable that a user will shake hands 
[ character / virtual ] within a screen for example, if a user shakes hands [ robot ]. By this, a 
user will do the virtual experience of the interaction with the virtual world in an output also in 
the input phase, and sense of reality improves. Moreover, the virtual character which exists only 
in a screen can be used, or the participants in a party can enjoy the gap in in the "real world" 
and "the world in a screen" with processing and outputting the input from the real world, and the 
virtual reality which cannot be experienced can be experienced in the real world. In addition, if a 
part of fictitious character is made to exist also in the party hall using a stuffed-animal suit etc. 
when using the fictitious virtual character which exists only in a screen, it is expectable that the 
increase of unexpected nature and the above-mentioned effectiveness increase. 
[0042] Moreover, the same output as the output unit which branched and installed the output of 
data processing and a synthesizer unit in the party hall also in control room 711 is obtained. If 
what changed the voice of a controller 81 into each virtual characters as voice data outputted 
fi^om data supply equipment will be used through data supply equipment that a controller 81 
takes a questionnaire in control room 71 1, or takes out quiz **** — etc. — the spectator and 
interaction of a party can be done mutually, and in spite of not being in the party hall, it can 
participate in a party indirectly. Moreover, if a large-sized screen is used as an output means, 
even if there is no special facility of a stage etc., each user can carry out a self-expression to 
the whole party hall using the whole body because I approach a mobile robot and have him 
project. Moreover, it is expectable by adding added value, such as making a mobile robot carry a 
drink etc., as a psychological effect that the more natural image which makes the image of the 
mobile robot as a "data entry unit" fade out of the consciousness of the participants in a party, 
and does not have the feeling of tension of "a photograph being taken" and "looking" can be 
photoed. 

[0043] Furthermore, it is expectable that the sense of incongruity to the increase of a sense of 
closeness to a mobile robot and a mobile robot existing in the party hall fades, and the image of 
the mobile robot as a "data entry unit" fades further out of the consciousness of the 
participants in a party by replacing a mobile robot in a virtual character and making it display in a 
screen. Furthermore, when a thing without feeling of a "mobile robot" serves as roles, such as a 
cameraman and a waiter, it is lost that third persons other than the participant in a party 
intervene in a direct party, and the effectiveness of the ability to make the participants in a 
party relaxing can also be expected. 

[0044] In addition, although said example was explained taking the case of the party hall of a 
stand-up meal format, the equipment by this invention is not restricted to use at the party hall 
of a stand-up meal format, such as use as a part of restaurant entertainment, use in various 
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theme parks, or use as a simulator. 
[0045] 

[Effect of the Invention] As stated above, according to this invention, an input means is attached 
to a mobile robot, by offering the means for "the element of a virtual world is embedded to the 
data expressing the real world", and "sequential processing and compounding an input and 
outputting it", he cannot choose a location, a user cannot be conscious of the virtual reality 
actually high quality soon which used the real world as the base, and it can realize in a natural 
form. 
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* NOTICES * 


JPO and NCIPl are not responsible for any 
damages caused by the use of this translation. 

1 This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated 
3.1n the drawings, any words are not translated. 


DESCRIPTION OF DRAWINGS 


[Brief Description of the Drawings] 

[Drawing 1] It is the functional block diagram showing one example of the equipment by this 
invention. 

[Drawing 2] It is the equipment configuration Fig. showing one example of the equipment by this 
invention. 

[Drawing 3] It is plant layout drawing showing one example of a data entry unit. 

[Drawing 4] It is the conceptual diagram showing one example at the time of using the equipment 

by this invention at the party hall. 

[Description of Notations] 

10 — A data entry unit, 11 — A mobile robot, 12 — Data input means, 121 [ — Bon Festival, ] - 
- A video camera, 122 — A microphone, 131 — An arm. 132 20 — Data supply equipment, 21 — 
A data supply means, 30 — Data processing and a synthesizer unit, 31 — Data processing / 
composition means, 40 — Data output equipment, 41 — Data output means, 411 — A screen, 
412 — A loudspeaker, 50 — The control unit of a data entry unit, 51 [ — Control room, 712 / — 
A table, 81 / — The controller (human being) of a data entry unit 10, 82 / — Participant in a 
party (human being) ] — The control means of a data entry unit, 511 — The video camera for 
the party hall complete view photography, 6 — Means of communications, 71 1 


[Translation done.] 


http:/ / www4.ipdl.ncipi.go.jp/cgi-bin/tran_web_cgi_ejye 


2005/03/09 


1/ J — 


* NOTICES * 

JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

1 This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 


DRAWINGS 


[Drawing 1] 


^10 

-^11 


63 


^21 



L 









\ 

t 


31 


62 


64 


61 


30 


51 

.50 


41 


40 


[Drawing 2] 


http:/ /www4.ipdl.ncipi.go.jp/ cgi-bin/tran_web_cgi_eije 


2005/03/09 


37 >K - f'D-?' 

I 


62t 

/ 


122 

.118 / 


^ 


114 623 


JiO 


> «2 


5141 . 
514Z < 
6149 
5144 
9145 


914 


ij 61 1 


^616 
519 


MI ^ b7^=r*/'^ ^^^^^ 


[Drawing 3] 


641 -V 




643 v^. 






211 
212 
9131 
91Z2 
9129 
)12* 



— r~ 


~i — 


'412 


http://www4j'pdl.ncipi.go.jp/cgi-bin/tran_web,cgLeije 


2005/03/09 


O/ O ' X — 


^4 



[Drawing 4] 

03 



[Translation done.] 


http://www4.ipdl.ncipi.go.jp/cgi-bin/tran_web.cgi_ejje 


2005/03/09 


mB^mmif (jp) (12) ^ H # ^ (a) UDmm&ms^ 

#^¥10 -208073 

(43)^:^18 H ¥fiE10^(1998)8fl7B 


(5i)inta* mm^ f i 

GO6T 15/00 G0 6F 15/62 36 0 

H0 4N 13/02 H0 4N 13/02 



1$H¥9-5243 


000005108 





c&amB 

3p«9ip(1997) 1^168 






mn m 












ft* 








^s^B^msm'yT.'T'mm^n 








fi^iijiuncsm«£E£ii^io99s^ «c 










(54) «a@>^J^B 


(57) 

h ( 11 ) ifi^Jjm (12) 
(?m0ll//cA^$gl (10) A^i^gl 0*>6 

SSi (2 0 ) iPb<0}j3ti:r-^<0mi. ' 

(30) rini • -^sav. at>ms. ( 4 0 ) fcai:^^- 
i. 


—10 
-•11 

•« 


Si 



41 


3 


40 



^5t 






(2) 

1 

r. 10 
s. 

m^T'-ntimomzoT-^t^^iiiRo. frsa^ 20 
a. 

[000 11 

[^lB©s-rss%^»i S!i<f©7'-^'*^ 

(0 0 0 21 

tt. ;^©<fc^)«cfe©*s*5. r^i&isi©*f** 
ici^htitcumco'pxTimimBb. -e©M^=&*)63&> 
s. cntt-ii^K:-'^•-^•^;^;^^^£^3^^l*«n■s«>©r 

iJ-iMiC-OiiXiihlK ^^•^OfJMvX^O-^-tJ 40 
-©SSt^rJiO. <S*r$^±-C^tiP>©-7-*-©g,t 

♦ + fi^t wns ^j)©r * 2.. 
[000 3 1 r||:?^-c-x(c{gstsi?©sig«: 
Si&jitf J iEB^BUftfiJ^tctt. fiil«t3&>©li»*>^ai 

i¥«ti-sfe©r*-&. 50 


|$igjpi 0-208073 

2 

(00041 Se>(C. igS((n;J<9 h*€&5*>©<i:l.-C 

xmyinX\,^i>'^OX. — jK(C*-i'3>3>hP-Jl' 
[00051 

[i6?s*5)g^L/j:^<!:-r^^i cne.©«^-c«. 
=ettW'vaji)iitf*t^^jiiittin©!!(i^A> e. isi-c^j 0 

^^ft©gPM5:'i:>Si Ufc. -e©fc«>. -e-©Si5M©*r 

fc. *y5«si5S©!^5e©^iS{cgs$n-c*j»). ftg 

iiSStt-Ctr5*JJS0f©S:ttM:-C*«:l,>fe©T*5®-c. 

mim.t 7 <b©®ai*siith-c u * ^ isi^©®*^ 

■6. 

[ 0 0 0 6 1 H^C G^Ji0Ka5«:*5l,»Ttt. r« 

rja#©H*i«i^rir^j &i*©m*3!>^ 
*5 fc©©. <KSi«:ll©+ -f -7 :S?)&S86 < K) . 

3^tSlfLt^^1SI!ii©tBSf?ffi*sj&lr>. i/c. 3^i«: 

(00071 :*:^tt. Rh©^@{cML/, ^ift^t 
l^t&^i©S«SftJ^<DM«I*»Bf*j8«-r(c||31-r-5 

i^^stt-r'5ci5rgfi«jfr5fe©r*5. tfit> 
i®©^«:!^s©spM^ci'4ji:»st-i*T. ffieesi 

ll«r(*SIT €.fcJ&{C -7 -:3^;-4ot:t i tc i'©3.— IffiJ© 
A*#S*>6©A*©®*litl»*Jt*6 

(000 81 

)B^-r5»fc«){C. jyT©^fiSit/c. T'-i'XJjujmrj: 
^. mfi2^l6tt©^»iK:*fi6-:Jl:fr. tUmT'-ii^^ 

©7^- ^ *J«ttXBfiga7=- 5? t «SIJ©|| 2 ©X- i 


3 

[ 0 0 0 9 ] J: oftftwtca. «T©a») r**. 

o^^r, V t^i:xv:mti7='-■^^J]^m^mxfcy'-- 
[0010] T'-i'^jj^m^m&ti^^v. vicmvx 

-s. 

coo 1 1 ) 

mismvi>i, si{c4jt>r. T^-^'A^^i^gi 

SiPjK-? f 1 1 tU:^v h 1 1 (cfffli-rsr-f 

ei 25r«i. jHt#S6 2. 6 3 ift"^ (^-af) ® 

^^xmtjtt. 2 0 tt7=- ^m^^ 2 

l*«i^.4J:5&*>©-C*5. f=-**DI-^Jil^3 
Ott. 7"-3f*0I • ^t£^©3 1 ^mX. a^#g6 4 
*>6©A*M#t7=-3'«tJ^S2 0*>6.©A*m#«: 

. l^it^. r-i'ttlx-^fiS#K*^^©A:tifi#=&r- 
^'Hi;b*S4 l^fflt»-cm;^Tifc©t?*.S. -r-i^A 40 
:^)^g©fKSiPJ^g5 Ottr-^r A:^F$gg©*IJi3f?g5 1 

fc©-C*0> l*JiCf7"-:SfA*#Sl 

■e©^-^^ffiM*©6 1 ^m.^xmiti>. 

[00 1 2] *ISSSW(c4sl,>T«. iwiatigso 

. if'-^'Aj^iisi ocrBiteicfx-a^A^^i ot 
Jni • -^^^gs o©rBitt3i<i#s?:fflt-»:^c:&^ 

-Cg^ L . i'^gg 2 0 i r- irai • -^fiS^g 
3 0©rafcJ:CF7'-a'ftIX-^fi£S^g3 0 tT^-^fffi;^ 50 


^ra^l 0-208073 
4 

iig4 0 i©isittm-^Titg^g^eir2.«fiij«:axo/c 
ifi. cn6>fl!^gi8i©ssKTttm^©swiSu*s-c*n 

t3;i<. gg5 0i$ggl 0©P5*5t.>tt^gl Oi^ 
g3 0©ra*ft-^-Cit«^^SC<!:t>T?tily. il 
g2 0 <b£^g3 ^QMhWi^^^ 0 6gg4 0 ^© 

[ 0 0 1 3 1 3 ;t^Wt?«r-5'«-^g2 

0. f^-$JlII--&fiRi®3 0. «fHiiPl?g5 0=S:-fn^? 

I • ^RS^g 3 0 *Sr- ^ C <b (C^ o 

Tf'-a»fii4$ig2 o©®fi©-aj*?t»^?rj(ttas 

<t5fr«ia^. ■7=-^fJ!)OI-^J!l^3 0*J«(HiiP^g=& 
m i C <!: tC J; o T*(IiaiSg 5 0 ©®f9©-aJ* /ctt^ 

• ^lii^g 3 0 i^f- m^^m:^^ h^*) um.^ 
^- ^rA^#s*^t?«fiS*«t^ 6ns c i tcairr 

■6. 

[ 0 0 1 4 ] a 2 a 1 ©«fiK^A«^^ftss-c«^ 

L:^ii^©-||j^«^-rilS^(iS@r<i&<S. a2(c«$ 
l»T. f^f^xxif^v h 1 1 Ott3v>F • 7^n-^f 1 1 

1 . m^uif^v \-pmw^ 112. lisii^ 1 1 3 . -fe^ 

1H14. txJr*^5S!l®^«l 1 5. v-^i/JdSPSI 

«i 16. A/D3>-'<-i'i i7*jicjfi 18. am 

1 lijj;c;6 2 l*5j:tJt6 2 2*J<i:Cf6 2 3€r 
M^. fcrT^:*-;^;^^ 1 2 1. •7Atf\^^if'¥im\JX\^ 
h.im^m 1 l*/M/-CA:tl$n/c$l|®(i^€:3v 
>K • 7=3-^^1 1 lt?«?*fU. 3-5'>F • f^a-dfl 
1 1 »3S^^^§0ffllfB#*S$^J^iPt5^»l 1 2. 115. 1 
1 6 (Cits. i^«in5j<^' h$)r<3PtS1«l 1 2«3'7>F . 
f=3-^<'l 1 l*i6^6nr< 2.*(Hiipm#«:»-:5tfi«i 
*1 1 3fcJ:Cf-b>-!>^l 1 ^^fmth. 
[00 1 51 ^86S1 1 3<!:l/-C«. ^SSOsJ^f h 1 1 
0©i^Sljt5«J©K*i. ef't^pf 7 12 1 ©^SbOjjC, 
F 1 1 0rtr©&g^iEfcL'<»*^7T>:}'->l'C^ 
M. iJiCfv-f i' 1 2 2©S«bn#'? h 1 1 0F<9'C©& 

^M^c^^^f5^se}^>#^f. */c. •fe>ifi i4i6>6 

mc7=-ftt. amjj^-F6 2 3*^6lB;^J3n^. 
3h;«;y5$i|fflltR«Sl 1 5«3-7> F • r=i-^l 1 \iJ^ 
62l6*ir < 4ftMai(i#«:S-3* ^7=**;/.^ 1 2 1 © 
fliS©A/^fc J:tJfM*.ffigf 43 J: D (Ofm^Vi 5 . 
brf^:t*^5 1 2 1 *i6A:'33n/cmWA/D3>-''t 
-an 1 7«c<to-c-ri;af;Uffi^(c^3ia. a^sK- 
F6 2 l3!p5ai:*Sn5. */c. ^-fe'*9M6«ll 1 6 

»=iv>F ■"f^-^w i*^6ii6nT< ^.jiJiaPM-^ 

(CX-tJtv^' i' 1 2 2©mi!S©A/^©*(l'iai5:tf ^. 
[0 0 1 6] v^i; 1 2 2*>6A;^3nfc(t^«A/D 
3>/>'-d' 1 1 8«;j:-ot:7=i^af;Mt^«c^gl3n; a 


m^- F 6 2 2 *>6Hi^sni. m^- f e 2 i . e 

22. 6 2 3*»6ffl*Sn. ^tl^njl©!K-F6 4 

K 6 4 2. 6 4 3i}^P>^J}^nfci§mt. <^=^- + 5 
i'^©CG*7^;l/2 1 l*Jj:C;iKe+i'7i'5i(D#^7' 
-d'2 1 2i<!:fetC7'-$ftn:7*Oi^5A3 1 2 IfcJ: 

c;7^-3'^ia:^ny7A3 12 zimii^xtp^mm 

i'??^ tfi^jtJ- F3 1 4€:ffil»-C;^5";->4 1 
IK. *fctt1f5>F#-F3 1 5%ffll>T:;^b--*4 
1 2{CtH;tl3tlS. 

C 0 0 1 7 ] C Cr. ffiffiK- F641. 642. 64 
3*>e>©A^Bji<t$(ISip:^ad/^A3 1 2 4?&ffli,>r4J 

ji^sim^siSBs i i{c.to-ciiotiip$nri)6. s^rt. 

CGt7'Jl'2 1 1. "S^^-itZ 1 2. :/oi/^A3 1 
21. 312 2. 3123. 3 1 2 4«j£JS(CJt;Drit 
3!§Bt^iS3 1 3«>A,£|B{S^S3 1 2i^{C^i&Sn 
St>©-r*S. SttnsK^r h©=i>Ha-^ (AIS) 8 
Itt. \^7':irt>jt^5 1 l*>6A:^$n-Si^an;i?7 h 
1 1 0<OmSm'^<O9kfm'f:~i''?-\^\::5 1 2 1 V^tj: 
*J6. +-5JJ-F5 I 6*Si,>ttv>;^5 1 7«:J:0^ 
F 1 1 OiSHSp-r*. +-5K-F5 1 Bfcic; 

5 14 1 iJ ic; tf":*-*^ 5 1 4 2 iJ J: 

i^M&mT'-i' 5 14 3 *ffii<^-cA*-4!iaP3- 

FI^:/ni^7A5 1 4 4«:S-:yi»-CctefeSIS:*!Hi^g 
5 1 3K:J:o-CiKfii8l=i- FtC^j^^n. F6 8 

lifi^tiitfitii, cere. jim5K-F6 8 KtSmil 

too 1 8] $liaiffl7'-if5 14 1. 5 1 42. 
5 1 43*Jj:af7'Pi/^A5 1 4 4. 5 145»ji:«(C 
ftO-C9^mi&!^Slb I 5«>^i^§Bt^lS5 1 4tceS» 

jissn44>©-c*4. ttto. aii#- fb 1 1 8 1 

©la. a<i!j<- F6 2 1 i 6 4 I ©la. MMiK- F6 2 
2<b642©BI. ji<i'J^-F6 2 3i6 4 3©rattjiiK 

o J:Cff^^M^©-r'<-C*J±iStf^gJ:{c 
?|«rt4©-C*n«. J^a^t^IgS 1 3 5r=&ISti 

[DO 1 9] */c. sri^ifswKfcci-ctt, wmm> 

•?X5 1 7«:^l,/c*«. *»l!SK:*jWS$(iai#m©i 
— !f-f>if 7x-;^ttA^-^Slj3- F^gi^'ny^A 
5 i44*jms-ctsfe©r*ntfj:<, Cti^tCilS 

|fll«:*JUt:»3>hn-5 8 ltt*r.irf-H^5 12 1 
*^e.©1f$R€rfei{C7^-i»A^^@l 0*iKliliPUTt»fc 


(4) ^^¥1 0-2 0 8 07 3 

6 

*S. *6?tlC*i$frl/t5 1 2 2*J<l:c;:^l^-^i!75 1 

8 1 *fi«L. f^-^rftii •^jai$^3 oij^p><om>^ 

^L/T^-^fT^ueS 1 2 2fc<fcc;:^fcf-*5 1 8 1 

[0020] tfc. Bflr^lJ6W«:M>T»3>FD~7 
8 1 tt?"- 3f A^S^g 1 0 ©a > F n - Jl/©;^«:tf -o TT 

il3 0 i b-Cf=-*A:^J^K?:ffi^/ct>©%fiefliTS C 
10 it?. 3>Fa-^8 ltt7'-i'A:tJi^l 0%3>F 

[0 0 2 1 ] 034jJ:Cf04». S2-C>j^$4a2.^% 
i L)tSIB«:fi::ft}^55©-'f-7^ ^ iE«-cffifl|-r 5 

5'A;^s^g©-iiist?«?:m-ri^gs-c& d . H4 ttc © 
^©A— 7^ ^ ^©-Utem^-rfiBHS-c* -s. 

(0 0 2 21 H3«:*JC>T. 7'-:fA:<Ji^gl o»a2 
©^S6ti!}?^ F 1 1 OtClf-f:*-*^^ 12 1. v^i; 1 
20 2 2. 1R13 1. tJfil3 25rff)S3-&/c4>©-C«>5, 
C©<!;t. H2>-!f 1 1 4«. ^tbmi^v \-0^tb^. 

tTf"**^^ 1 2l<D^nu^^v F 1 1 OrtT© 
fig. t:^:ttljt'y l 2 1 ©SS6o.-Jf ■? F 1 1 Ort-C© 
lajt ©4 o©x- *s«feai-C 1 5 J: 5 5 

t-rs. bff^:^*^?! 2 i»ab?>*>D«>4'f »; 

:/U-F=&fT-9r*s<. H4{C*Jl»r. 10 1. 102 

it6i>K.miSm7 1 1;&>6$<I@P^S5 0 1 . $q@P^5 
0 2 ?:ffll>T3> F O-^ 8 1 CC J:-oT$lItaiSnrC> 
30 4, 

[00231 y^- ^A^mS. 1 0 1 (Dij ji -71211*$ 

1 2 2 1 ifiP>7<^^titcm^is^lf^mt^ 
f*- f 2 0 1 *>6 ©A* <!: . -r- tSrftll • ^itS 

SgS30 i{cj:or^fij$n. xi'y->4 1 1 ifcj: 

Cf;^b-/74 1 2li>^mt}$tii>. usee. f'-tSfA 
:^ilgl0 2©*y7l2 12*jJ:i:;-7^fi' 1 2 223&> 
6A*$*l/cBi<^fcJ:CJfWi. 7^-a'fii#ilg2 0 2 
3&>e>®A*i. f=-^'fto:-^iS^3 0 2{cJ:or^ 
ESSn. ;^i"J->4 1 1 2fcJ:i:f;^fc:-*4 1 2 2*' 
40 P>W;t)Sn5.. r-i'^«g2 0 lt><fci:f2 0 2*>6 
©A:^JB. it>KHRfi[«*'f 5i'$©CG*-fJb2 1 Ifc 
J:CJffieffi=^i'7i'i'©§^7'-f 2 1 2<l:0;«4. $ 
fc. •f-fttn:--^fiSgg3o©gsiiST«rat>rj:i,». 
#li©s^«^?^stcsg^n/c^^7=^3^*y 7 5 1 itcj; 

•oXfm^l 1 IrttcSgStifc^xiff^ixtrs 1 2 1 
(CKl/tflSn. 3>FP-7 8 lW^©l!fc«<&S'5t*J6 

7=-^fA*iigi 0 1. io2immri>cti!fix^t 

(0 02 41 g|4{CiJt,»T. b^f'3^*>7 12 1 itci 
50 ^T»e.n/t!ft«tt. ^Wio^^y i-lOl ^r^KS+i-^ 


(5) 

7 

^^21 1 1. ^&ti^v h 1 0 2*lKe=t= + 7^^?2 

1 1 2{cgdft*SJg'C^Eg1ftl!ii^iasti. 

>4 1 1 l{cffi;^j$n-S. PfilCC. 1 2 1 

2{CJ:-5-C^e,nfc!!(i^tt. igJS&PjJC-^ HOI i:m. 
^^yi;^2\ 1 1. i^8i)a5j<7 H 0 24<Ee=1ri'7 

i";->4 1 1 2(cW;tF3n^. P«(C. ^-^^^122 
l*'^©A:^7«W^7'-5'2 1 2 1 i^]5S$*XT:^t'- 
*4 1 2 1 (C, -7^ if 1 22 2*>?>cr)A*«^^7'-af 

2 1 22i^Jfl{StlT;^tr-*4 1 2 2{CtB:*3$nS. 10 

^'•f 7i'$©3^>[7cCG*f'Jl/2 1 l*JniLfcfe©* 

a«i)j2^fc±-c. «c;2^Xjt7'-a'«:^-r-5it»5:^r 
[ 0 0 2 5 ] 0:43. nmmm^^\.^x\mm.i 1 1 

*-'-t-7"^^©2|!i(Ci3:§L/c*S. ^S7i 
B-fe>1f- 1 1 4*J<J:Cfa{ij}?- F 6 2 3 *ilitT(r>A: 

[ 0 0 2 6 1 $yfc. isrfaiiisWKfci^riif'-f so 

SI 2»t^7':*-*^7 1 2 lfcJ:C/v-<i; 1 2 2/cWr 
*o/c*s. *|%Bjcr)7=-a'A:^j*©ttcn6(ci!B5fe© 

©4 itt:^d''j->4 1 l*jJ:cf;^br-*4 l 2Wii'C 
*^/c*s. *^i5g©7'-tS'tti:fc»#K«cfi?.{c|iS^fe© 

?>12 3. r-4'ai;^*StOr?KB^r-<;^7-U-f4 
1 3*ttlit*. cc-c. • h 1 2 3©W« 

tCliSES?^ X 7" U 4 1 3 ©Ift^WS J; ^ (c iJ 

<. T-^?»DX'-^figsg3o©ai;»3'&^iie^3-a-. ?gs 40 

f yX7U'f4 1 3{C;:^i'y->4 1 1 iplUIS^^Ift 

fP'Cfi5< Ci*5-Ct-5l/. Xi";->4 1 i±©<Ee+ 
+ ^ ^ ©mtC^S t 5 C <!: fe-C 1 5 . 
[002 7] ^{C. 7^-^rA*#©*jJ;c;r-iJai;^#^ 

tt:^d:tvc<!:k;a.«-r?>. ^SIjd^v H 1 0 

tc, Kl 3 l©Rt>Otc. !g5*5SniJ:^Aj:^iS^Ofc 
ISeI 3 3?r{^M3^. A^^gi brtt;'3-fe>1f*ffl^ 
b. r-^f#t#i^g2 0*i6Kl 3 3©*«SPfflf'-5«*i 50 


0-208073 

8 

•§$gS2 OiJ^hntLm 3 3©*!l1iiPffl^-i'K:S-:Jt>-C 

3 3(cffi;t)Lr^2>c<!:r. reE< 
sn««-€-5tta?:i/T (c©^»;^e";->4 1 1 
«cffl:^$tiS) 9ii<Ji»)igu-c<sj ttiimmm^ 

ife-CtS. &*J. C©«^7=-^JjjnX--^/iS^g3 0 

(002 8) 3 |irfei^t^(c*jc>r «-r- i^tt-^ 

t»«iatg*©* i v» 6© 2 afet±?:{iir i>ti« 
<fc<. ^§at^gK:iESfe©'C«^ci». J: -sr. t'-:^ 
^Slg2 0©a;t»tl/-Ctt. *e.*iD&ffi^bfc7'- 

tcgt:fS7"-^f«j^g2 0©a;t«Jcn6(cK-5fe© 

•rif"- if ttiKfi+ + 7 ^'©C G*f"Jl'43 J:c;{R«+ 
•t'^e'^©W?&f'-5'-C*oyt3!»i. ^'-^J'fit^gAi^ 
ait)ti^-iitl>Xit. cn6(cEi*>©"Ctttci,>. 
( 0 P 2 9 ] S 6{c. l9ffilli6W«:*jt»t:. T'-nmx 

■^g2 o*i6©A*^B(^i/-en6*Jnx • -^fisua 

*n«J:<. «mL/ytt>©telKi!2>Stt*l>. S6«c, 
i*. m«, iJS^UfiWJcMirtf'^^^^i 21* 

OrCfv-^ i7 1 2 2©§}S?:^l6n*Kf h 1 1 0©m®i 
I^D< milg©A/WW31-{ct^7^:J-*y7 1 2 IfcJ: 
1 2 2©S)ja€tft)^j:t»Citc:r4a«. mii& 

mi 1 5*jj:cxi 1 6. mmm^—f 51 42i5^zf 5 

14 3. 3V>F- T'a-^^'l 1 l«#BSpJfig-e*5„ 

SA:. m^ii. mi&mmm(oj:^i,cw&m.7 1 1*>?>^ 

[0 03 0] 3^tc, f^-^iA^)t^gl 0*>6©a:ti^ 

mm.i> ^ 5 ^d:fflm#i8^$)j«ipiig 5 0 om^mt b 

xm^i. f'-a'X^Sgl 0*»67^-K/<r.47§hfc 
7=- ^f^Wffl b-Cf^- A;t»^g 1 0 ©fJj@i* J; OgJ 

{cT2>cifc-cts. fijai^gs o*i*(iai^5*t 

lii LX\i. t^xB. ^fbu^y h 1 1 ©Sb*-^. r- 
iiXtJ^Wt 1 2<D^Via^^y h 1 1 rt«:*jWifi[gfcJ: 


(6) 

9 

«r X 3^ -©f^ito 0 {CK|#EI C i nJtg-c* 

4. ID 

[ 0 0 3 n s ws^^mMmici5\,^x^i^JJm. i 
oiic{^mtiyi7'*^^'y 1 2 i*jj:i:^-7-f 1 2 2» 

■en2^-roffli»5c itc-Ti ci«c<tD. isMfefcicf 
[ 0 0 3 2 ] S e,{c, iaaiUiii^{c*ji,»t:»7"-3'«^ 

^5 0©lSt»r-i?A:^i^i o©Sfei-aL/-ct»/c 20 
SI o©a(c{s»ri«>©rB<t<. ^^/jgjSttSWfw 

-^Kh^SifilOl. 1 02. 103. 104©4^ 

-en-eni# (2o*jj:c;3 0) . ^f-mt^mifiA 

111. 4 11 2©2^. t'-^'A:'^^©*!!®!!®*! 
1^ (5 0) <bl,»^#^*>pJ|g-C*5. C©Ji^. m 

1 0 i(cRi-riffi;^7^;^^";->4 1 1 i©£¥^cc. 
f"-3»A:t»S?gi0 2(cH-rita;fj?:Xi7y->4i 1 
l©e*»{c. ■f-3»A:t»^i 0 3{cH-r6m;t74:^ 
i";->4 1 1 2©S2|£»tt:. f'-^fA^Sgfil 0 4{c 
K-rsai:^*;^i";">4 1 1 2©:&*^{c. -eh-eh 

^fa^ii^Ti,»n«. 3>f-a-7 8 l*5fil^^j:J8** 
ffl-3-ct'©Hi:;j^a*$-&S36i*^it.scit>pJiigr. 40 

[003 31 S6tC. |JI2||iS6W«:*jl,>-CBA— f 

m 1 -^mivh-yfcifi. *^WJc n(cK.5ib©rB 
^:t». r-^A:tiSg*i^ttpJ«6-C*0. 1»<-3>*>©JS 
0fr3^lliSS©A*?:?f^.ci*i^iee-c4>s©r. t^i 
tiH?Elli6««lil^©-'<-f^ -f 20(701. 7 
0 2) fflSUTfet^T. ;^i'';->4 1 1 l(CA-r-< 
^7 0 1©«^^. Xi";->4 1 1 2(C>'N-— f 
li7 0 2©fiH^5:!!l5Lai-r.Ct4>^fie-C*^.. *fc. go 
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7©+ + y 3 >J&m»^81)n;K 

htc-fe^-iH 1 OSft/i§-&:fc*s. ±S©tfHi- 7^-* 
A*|gB©«W@S?»{«5^ Lfc «>©tclS-5 fe©-Ctt«t 

[ 0 0 3 4 ] S 6{C. 8)ll^*i6©3i>C5cJ^Ji3£fi3Ktt 

iiriB©i^3^5>j®fiyt!ia«:iE4fe©-cttttt». m«. 

-:5'A*^©<tUT2^©l^f'jf*y5*ffll»n«. ;^ 

• ^5^^c^')©cl[g^c^©^^•-f• ^^©7^-f **e. 
jj^UJMf r iiat^gtcs^r *j # t'- ^m^kmis^ h 

-n^iiifom^icmmwi^m^tbaif^y mo 

<fcc;i^Sba«K-^ Kc^<i«|5&#W^©3yj^K:-e©S<i 
fg©a7n©}6Sfflg^|Sl±$-&5C<!:*i-C#i. 

[ 0 0 3 5 ] 3 e.{c. mMim^mi<cisi.^-xntmmm^<D 
^jjicnri>mj:- -^mmicmvx ^mnm^im 
7ci/rigiS+ + ^© 3 i;5c*f^jv*a«wi*. -tn* 2 

*l%WK:*jW-sli^m-^©A:^K:*t-rsft]X-^ 

fi)u&a»cti(ciE-s.fe©-ctt&(,». m«. sfj^tctSD 

r. A*$n/c!Wfe©Ji«J!Scaa«:2^J?7c©igS+i'7 
^i:«aj»%ffi5©|ii»*>6«}lfflt-r r^nn^v hm^a 

[0 03 6] $e>(C. tiiiBll;te«SJ{C*Jt>-C«f'-3r©J!)P 
X • ^^mS3 0-C©^^m-^©^fiX«*^c6 5 

r. a— tf©^H?r)t)0X-i-'5CifenItgr*i. m 

«. B>rieii*S0a«:iii,>-c^i";->4 1 1 1©^*^>. 

:&^|i^K:-tn-eni— {f 8 21. 8 2 2 5:7 7 T'Tiftl/ 
ffll/. *6*it«>igBt^g3 1 2K:S^T*i(,>A:»* 
©^^^-^?:ffi(,ira— !f 8 2 1 . 8 2 2©^K?:j3n 

xi/-c;^f-;^4 1 2 1. 4 1 22{cili:^-rn«. 
[ 0 0 3 7 i nn^jmic^^^x\tm.'>r ^ 

S+ + -7i'3»©^»Cti«:K'5*>©'C«:&C>. m 
. <5iA*^©^Sffi+ + 7 i' 3f 5: A:^ S *ifcii*^{c#fiS b 
ri<i#j <!:U-cx^';->4 i i{cm:^-r-5cife-c 
^2.1/. {5iA*^©<5i@+i'7^'3'*A:tiSti/ci!*««c^ 
fiKL r>'N--f-^©#tti^j iLT;^ij";->4 1 item 

8 2iiss++7d'3'*s;^i";->4 1 ncmmitVi 


TctiFik I^OA^^a^^-fA/. >a>+•»' 
[ 0 0 3 8 ] ifc. frffiHtt^Kfet^-CttT^-f 
SI 0l*Ma^S5 0*fflt>t:3>hP— 58 

Si^i. ©iaKC!iJsyifcW©-fe>i^t4i^Kia#? h 
1 1 o^cfi*ii^-c. f^-a^A;*?!!!!! OKS^fife* 

[0039) J£<±©||Jtp»K:in«. 7=- ^>J]^^ 

h<!:«bfc{caS(CS()tB5Ci*s-C*i©-C. tif*© 

S©J:i>A**S5Ci-?'. SViCt^y^T^yjl'Ch 
T&&©m fflJ^©^)«:J:iA**W^>Ci. ^ 

[ 0 0 4 0 ) S/c. r^nts^^^^^T^-iJtcissi 

iftii©s3R*a»iitjj i(,>^:&«f(cj:o. aa^^fti 

€) :i-tf ffliJ-Ctt V- ;^ - i/j: t-©JSHids:}qg-c 
tSWJ *a«&iitfCi*J-Ct5©TJ:»33il|{cifil,><SS 

^Jnx • ^fiX(cs-ri*jia©«*5Sij«3ti.s c itco: ^ 
[004 1] tfc, rAt>m-^xmx--^fS.um:hbx 

©?fS65:l§I»3SHtSCi*5aJtgi^c2>. $6(C. ^S&P 
h{cA:*l*S*f=tR$-t*SC<!:-CA*#©*sSK::x 


(7) !HFSi¥10-2 08 07 3 

12 

-• !f©>a<^c??^tr4ci*snIlfe^^cs©-c. 0dji«. 

©ta5f¥ffl4^©A*SPg«:45t»-C fe^wsrr ^ c t 
A^%flniL.rm;^j^SC<b'C. ^^•-f•-^©#3!)0«^^:•^ 

r=iillS5?j i rxi'y->©ct3©ift»j i©¥t 
it«:#SE-r s5?s©<Sie=<= + 5 f ii^jctt. 

[0 04 2] T^-rSfJlII-^fiS^Ott^'&^fe 

20 ■en©{gie+i'7^'^rffl(c^i/fc*>©^-f-^»fit^ 
-^x^Hiuyto <civ«-7^ ^ ©astffli:f^ffl*s«t 

#J8!^cS«!&5«t<T«>. ^«iojK9 Kc2-:Jt,>T|!llibT.. 
fee. ^ C iV^^<D3L~int^-<-7~ ■< ^±WiCi^±^ 

©#ttIflifi«:W:tm€.Ci«:j:»). a— r^©#Jo# 
fcS©SlK©**i6 r7"-i7A:^j^gj tUT©i^ttn 
^7 h©-f^-s^4lins<±. rjii;3nrc»ij 

[0 04 3] ^tbu^iiy + 

#&-rsc<!:K:*fr^^faJS*i»n. ^t-7^-<©#]!)p« 
40 /i:^©^ii©tp*^e> r7=-ajA;*jsigj <ti,T©^«)a 
;J<f h©-<^-i^*JJ:»)-Sf|Enic:<!:*5«^-c*S. 
S^tc. ri^Jftasjet? hj tl,»^»®»«rmc&i,>«>©3»i 

-^ ©#ftl^m©mH^*ii^S/^•-7^ (c:n-A-r 
5 C < /j: O . ^'t-T' -f ©#IlD^/ct >; ^ y ;^ 

[0044] tj:is, m^smmm^t:^L■AB^<o^<-7'^^ 

h7> • r-^y>f-©— aJ<i:L/r©f<Jffl*^.l» 


13 

[0045] 

mil ^mtc^^^<o-mim^t9smm 
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10" 7'~^Xt}mM.. 1 l-i^8liDjK7 12-7= 
-ifA*m 1 2 l-t7=:*-*^7. 122 -v-f 
13 1-R 13 2-i$S. 2 0 ■7'' 
2 l"-f-5'ei#^l8. SO-T^-tJfDnX-^fi!^ 
Sl-r-irJnX-^JiSS^K. 4 0-7=-:ftt;t»$^ 

B. 4i-f*-a'm*#s. 4 1 i -j^fj-x 4 1 

2-Xb--*. 5 0 •••7=-5'A*$gB©«lffl«^. 5 1 

RJg^ffi^^7==^*^^, e-iimm 7ii-$ifii8i 

7 1 2-f^-:/n'. 8 i-f'-^fA^JiaEi o©3 
>ha-5(Alffl)> 8 2--'<-f-^©#ap«0Mffl) 


[@li 


It 


11 


63 


Ti 


5 


1 1 5=^-5«i^^ja 1^21 


J 


31 

1 


41 


62 


64 V- 


61 


30 


40 


(9) 


'mSW-l 0-208073 


914 


S14I , 

6144 
8149 


1l» , — 

, 117 







421 


3T>K fa-J' 





H til 


«i4 a» 


»«3 


Iifll4t-K 


S13 



441 >^ 



-«42 



643 

attK-K 



S11 w 
212 ^ 
9121 •sm^ 

S11S *SMr 





4139 ."^^ 






""1 


^418 



